{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lsCK4KDHw4LK"
      },
      "source": [
        "# **Why JSON mode?**\n",
        "\n",
        "You can guarantee that a model's generated output always adheres to a specific schema so that you receive consistently formatted responses. For example, you might have an established data schema that you use for other tasks. If you have the model follow the same schema, you can directly extract data from the model's output without any post-processing.\n",
        "\n",
        "To specify the structure of a model's output, define a response schema, which works like a blueprint for model responses. When you submit a prompt and include the response schema, the model's response always follows your defined schema.\n",
        "\n",
        "# **Objectives**\n",
        "\n",
        "In this tutorial, you will learn how to use either OpenAI SDK or Vertex AI SDK in Python to generated structured outputs via the Llama 4 Maverick fully managed model on Vertex AI.\n",
        " See here for more info on using the [OpenAI SDK with Vertex](https://cloud.google.com/vertex-ai/generative-ai/docs/migrate/openai/overview#:~:text=The%20Chat%20Completions%20API%20works,the%20Google%20Gen%20AI%20SDK.), as well as recommendations on when to use OpenAI SDK vs. Vertex AI SDK.\n",
        "\n",
        "We will use sentiment analysis as an example use case, you can replace it with a different structure that's right for you.\n",
        "\n",
        "# **Setup and Relevant Links**\n",
        "Llama on Vertex AI (fully managed): https://cloud.google.com/vertex-ai/generative-ai/docs/partner-models/llama. You may also need to accept the EULA to continue.\n",
        "\n",
        "Official docs from Vertex on structured outputs/JSON mode with Llama coming soon.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aEOMDh39uMYV"
      },
      "source": [
        "# **Setup and Relevant Links **"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "id": "2sHCLvoCw6TA"
      },
      "outputs": [],
      "source": [
        "import sys\n",
        "import IPython\n",
        "\n",
        "%pip install --upgrade --user --quiet google-genai\n",
        "app = IPython.Application.instance()\n",
        "app.kernel.do_shutdown(True)\n",
        "\n",
        "if \"google.colab\" in sys.modules:\n",
        "    from google.colab import auth\n",
        "\n",
        "    auth.authenticate_user()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {
        "id": "JG5BoLBdw5je"
      },
      "outputs": [],
      "source": [
        "import os\n",
        "\n",
        "project_id = \"[your-project-id]\"  # @param {type: \"string\", placeholder: \"[your-project-id]\", isTemplate: true}\n",
        "if not project_id or project_id == \"[your-project-id]\":\n",
        "    project_id = str(os.environ.get(\"GOOGLE_CLOUD_PROJECT\"))\n",
        "\n",
        "# run gcloud auth print-access-token from terminal to get this\n",
        "access_token = \"\"\n",
        "\n",
        "location = os.environ.get(\"GOOGLE_CLOUD_REGION\", \"us-east5\")\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DsRB381L0bwb"
      },
      "source": [
        "Defining the format we want to return..."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "id": "7CyBOejOz8iR"
      },
      "outputs": [],
      "source": [
        "response_schema = {\n",
        "    \"type\": \"ARRAY\",\n",
        "    \"items\": {\n",
        "        \"type\": \"ARRAY\",\n",
        "        \"items\": {\n",
        "            \"type\": \"OBJECT\",\n",
        "            \"properties\": {\n",
        "                \"rating\": {\"type\": \"INTEGER\"},\n",
        "                \"flavor\": {\"type\": \"STRING\"},\n",
        "                \"sentiment\": {\n",
        "                    \"type\": \"STRING\",\n",
        "                    \"enum\": [\"POSITIVE\", \"NEGATIVE\", \"NEUTRAL\"],\n",
        "                },\n",
        "                \"explanation\": {\"type\": \"STRING\"},\n",
        "            },\n",
        "            \"required\": [\"rating\", \"flavor\", \"sentiment\", \"explanation\"],\n",
        "        },\n",
        "    },\n",
        "}\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pRym0HCpt99W"
      },
      "source": [
        "# **Now, generate JSON output**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wSTscqE-wxyd"
      },
      "source": [
        "First, with OpenAI's SDK..."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "yU25L2KDpMSG",
        "outputId": "c7f8f0c7-3262-434b-f713-8f002d97dd0f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "{'reviews': [{'text': \"Absolutely loved it! Best ice cream I've ever had.\", 'rating': 4, 'flavor': 'Strawberry Cheesecake', 'sentiment': 'Positive', 'explanation': \"The reviewer uses the phrase 'Absolutely loved it' and states it's the 'Best ice cream I've ever had', indicating a very positive sentiment. The rating of 4 out of a presumed 5 is consistent with this positive sentiment.\"}, {'text': 'Quite good, but a bit too sweet for my taste.', 'rating': 1, 'flavor': 'Mango Tango', 'sentiment': 'Negative', 'explanation': \"Although the reviewer starts with 'Quite good', they follow it with a negative statement 'but a bit too sweet for my taste', indicating a mixed sentiment. However, the rating of 1 suggests a strongly negative sentiment, which is inconsistent with the text. The sentiment classification based on the text would be 'Mixed' or 'Neutral', but given the low rating, it leans more towards being negative overall.\"}]}\n"
          ]
        }
      ],
      "source": [
        "import json\n",
        "import openai\n",
        "from openai import OpenAI\n",
        "\n",
        "# Setup client\n",
        "base_url = f\"https://{location}-aiplatform.googleapis.com/v1/projects/{project_id}/locations/{location}/endpoints/openapi\"\n",
        "client = openai.OpenAI(base_url=base_url, api_key=access_token)\n",
        "\n",
        "prompt = \"\"\"\n",
        "  Analyze the following product reviews, output the sentiment classification, and give an explanation.\n",
        "\n",
        "  - \"Absolutely loved it! Best ice cream I've ever had.\" Rating: 4, Flavor: Strawberry Cheesecake\n",
        "  - \"Quite good, but a bit too sweet for my taste.\" Rating: 1, Flavor: Mango Tango\n",
        "\"\"\"\n",
        "\n",
        "response = client.chat.completions.create(\n",
        "    model=\"meta/llama-4-maverick-17b-128e-instruct-maas\",\n",
        "    messages=[{\"role\": \"user\", \"content\": prompt}],\n",
        "    response_format={\"type\": \"json_object\"},\n",
        "    temperature=0.1\n",
        ")\n",
        "\n",
        "product_reviews = json.loads(response.choices[0].message.content)\n",
        "print(product_reviews)\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ckdF273K0g7E"
      },
      "source": [
        "Now with Vertex AI SDK"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "OnfgZyHZr9gl"
      },
      "outputs": [],
      "source": [
        "from google.genai.types import GenerateContentConfig, Part, SafetySetting\n",
        "\n",
        "from google import genai\n",
        "\n",
        "client = genai.Client(vertexai=True, project=PROJECT_ID, location=LOCATION)\n",
        "MODEL_ID =  \"meta/llama-4-maverick-17b-128e-instruct-maas\"\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Z_ka6SP90Am-",
        "outputId": "69d91ee1-96d4-44f9-d22d-c4f4a9887e82"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "[[{'explanation': \"The reviewer used the phrase 'Absolutely loved it' and stated it was the 'Best ice cream I've ever had', indicating a very positive sentiment despite the rating being less than 5.\", 'flavor': 'Strawberry Cheesecake', 'rating': 4, 'sentiment': 'POSITIVE'}, {'explanation': \"The reviewer described the product as 'Quite good', but expressed a negative aspect by stating it was 'a bit too sweet', aligning with the low rating given. The negative aspect outweighs the positive, leading to an overall negative sentiment.\", 'flavor': 'Mango Tango', 'rating': 1, 'sentiment': 'NEGATIVE'}]]\n"
          ]
        }
      ],
      "source": [
        "prompt = \"\"\"\n",
        "  Analyze the following product reviews, output the sentiment classification, and give an explanation.\n",
        "\n",
        "  - \"Absolutely loved it! Best ice cream I've ever had.\" Rating: 4, Flavor: Strawberry Cheesecake\n",
        "  - \"Quite good, but a bit too sweet for my taste.\" Rating: 1, Flavor: Mango Tango\n",
        "\"\"\"\n",
        "\n",
        "response = client.models.generate_content(\n",
        "    model=MODEL_ID,\n",
        "    contents=prompt,\n",
        "    config=GenerateContentConfig(\n",
        "        response_mime_type=\"application/json\",\n",
        "        response_schema=response_schema,\n",
        "    ),\n",
        ")\n",
        "product_reviews: dict = response.parsed\n",
        "print(product_reviews)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wC4ci2UEF9KB"
      },
      "source": [
        "# **Congrats and conclusion**\n",
        "\n",
        "You've successfully built a sentiment analyzer leveraging structured outputs via Llama 4 Maverick using the OpenAI and/or Vertex AI SDK!\n",
        "\n",
        "# **Cleanup**\n",
        "You can perform the following cleanup to avoid incurring charges to your Google Cloud account for the resources used in this codelab:\n",
        "*   To avoid unnecessary Google Cloud charges, use the Google Cloud console to  delete your project if you do not need it.\n",
        "*   If you want to disable the APIs for Vertex AI, navigate to the Vertex AI API Service Details page and click Disable API and confirm.\n"
      ]
    }
  ],
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
